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ABSTRACT 

The  majority  of  accidents  and  incidents  in  aviation  can  be  attributed  partially  to  poor 
decision  making  and  judgement  strategies.  By  gaining  an  understanding  of  aircrew 
decision  strategies  used  in  the  operational  environment  of  the  cockpit,  it  may  be 
possible  to  improve  decision  processes  and  outcomes  for  both  expert  and  novice  pilots. 
Naturalistic  decision  making  (NDM)  is  proposed  as  an  intuitive  decision  strategy  used 
by  experienced  pilots  to  make  operational  decisions.  The  cockpit  is  considered  a 
naturalistic  environment  due  to  characteristics  such  as  experienced  operators,  midtiple 
players  and  teams,  dynamic  conditions,  shifting  and  competing  goals,  high  risks,  time 
pressure,  and  ambiguous  or  missing  data.  NDM  strategies  focus  upon  situation 
assessment  and  the  serial  evaluation  of  decisions  through  mental  simulation.  Decision 
aiding  to  improve  situation  assessment  and  decision  training  to  impart  awareness  of 
the  limitations  and  weaknesses  (heuristics  and  biases)  of  human  decision  processes  are 
both  aimed  at  improving  the  overall  decision  performance  of  both  expert  and  novice 
pilots.  The  overcdl  aim  is  to  improve  aviation  safety  and  reduce  the  number  of 
accidents  and  incidents  due  to  poor  decision  making  and  judgements. 
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Executive  Summary 

Judgement  and  decision  making  errors  are  the  primary  factor  in  over  50%  of  general 
aviation  accidents.  However,  ttirough  appropriate  training,  decision  skills  may  be 
improved,  thereby  reducing  the  number  of  decision-related  accidents.  This  report 
examines  both  the  theoretical  and  practical  aspects  of  operational  aircrew  decision 
models,  before  examining  various  methods  such  as  decision  aiding  and  decision 
training  to  actually  improve  the  decision  skills  of  both  expert  and  novice  pilots. 

It  has  been  discovered  through  several  studies  that  experienced  operators  in  their 
operational  settings  (including  pilots)  make  many  decisions  using  intuitive  rather  than 
analytical  strategies.  Naturalistic  Decision  Making  (NDM)  is  one  such  style  of  intuitive 
decision  making.  Naturalistic  decision  making  describes  how  experienced  people  make 
decisions  in  dynamic,  naturalistic  environments,  under  conditions  of  time  pressure, 
d3mamic  goals,  uncertain  cues  and  high  risk.  All  of  these  characteristics  are  found  in 
aviation  environments.  The  initial  sections  of  this  paper  examine  the  principles  of 
NDM,  detailing  theoretical  perspectives  that  explain  how  the  strategy  works,  and  how 
such  skills  are  acquired.  Intuitive  and  analytical  decision  making  strategies  are 
compared  and  contrasted,  highlighting  the  appropriateness  of  intuitive  methods  to 
operational  settings  such  as  aviation. 

Analytical  styles  of  decision  making  are  appropriate  in  certain  situations.  These 
situations  are  usually  those  where  the  decision  maker  has  specific  goals,  minimal  time 
constraints,  and  a  complete,  correct  set  of  information.  When  these  conditions  are  met, 
it  may  be  possible  to  make  an  optimal  decision.  However,  these  conditions  are  rarely 
met  in  operational  aviation  environments.  In  such  naturalistic  environments,  intuitive 
styles  of  decision  making  (such  as  NDM)  are  more  appropriate.  Indeed,  these  are  the 
descriptive  styles  of  decision  making  employed  by  most  experienced  operators,  such  as 
aircrew. 

Recognition-primed  decision  making  (RPD)  is  one  model  of  NDM  involving  decisions 
for  which  alternative  courses  of  action  are  directly  derived  from  the  recognition  of 
critical  information  and  prior  knowledge.  These  alternatives  are  serially  evaluated, 
without  the  need  for  the  comparison  of  options.  Decision  makers  compare  current 
events  with  previously  experienced  events  and  known  rules  stored  in  memory, 
recognising  similarities  to  help  them  select  appropriate  reactions,  judgments  and 
decisions.  Hence,  the  name,  recognition-primed  decision  making.  There  are  three  basic 
phases  to  RPD;  1)  situation  recognition,  2)  serial  option  evaluation,  and  3)  mental 
simulation. 

Because  naturalistic  environments  are  dynamic,  the  decision  maker  must  continuously 
reassess  the  situation  in  order  to  reassess  the  appropriateness  of  situation  models  and 
hypotheses.  Once  the  pilot  understands  a  situation  an  acceptable  course  of  action  is 


often  easily  identified.  Thus,  situation  assessment  and  awareness  is  a  crucial  and  time 
consuming  process  for  intuitive  decision  strategies.  Situation  assessment  is  also  a 
prerequisite  for  good  decision  making,  correlating  positively  with  decision  accuracy. 

Mental  simulation  is  the  final  process  carried  out  before  (or  during)  a  decision  action  is 
implemented.  The  decision  maker  evaluates  the  course  of  action  by  acting  out  the 
decision  in  his/her  mind,  imaging  how  a  sequence  of  events  might  unfold  within  a 
given  context.  The  simulation  often  includes  the  successive  steps  to  be  taken,  the 
potential  outcome  of  these  steps,  the  problems  that  are  likely  to  be  encountered,  and 
how  these  problems  can  be  handled.  As  a  result  of  the  simulation,  the  decision  maker 
either  rejects,  modifies  or  implements  the  action.  Because  NDM  emphasises 
satisfactory  rather  than  optimal  decision  outcomes,  the  mental  simulation  is  generally  a 
rapid  procedure,  acting  as  a  go/ no-go  check.  If  time  is  not  available  for  a  complete 
mental  simulation,  the  decision  maker  will  simply  implement  the  decision  action  that 
experience  has  generated  as  the  most  likely  to  be  successful  and  make  subsequent 
changes  as  necessary  to  maintain  a  satisfactory  outcome. 

The  importance  of  expertise  and  experience  is  explored  throughout  the  paper,  and  the 
differences  between  expert  and  novice  decision  making  are  presented.  It  appears  that 
only  experienced  pilots  are  capable  of  implementing  RPD  due  to  the  reliance  on 
memory  traces  and  pattern  recognition  gained  through  experience,  or  possibly  through 
training. 

The  final  section  of  this  paper  addresses  the  improvement  of  decision  strategies  (and 
outcomes)  though  decision  aiding  and  decision  instruction,  and  examines  how  some 
areas  of  NDM  and  RPD  may  be  able  to  be  taught  to  novices  and  inexperienced  pilots. 
Although  there  has  been  little  work  done  to  formulate  teaching  strategies  for  NDM  and 
RPD,  some  ideas  are  presented.  It  appears  possible  to  teach  novice  pilots  at  least  the 
basics  of  NDM,  which  they  £ue  then  able  to  develop. 
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1.  Introduction 


Over  the  past  20  years,  judgement  and  decision  making  errors  have  been  a  causal 
factor  in  the  majority  of  general  aviation  accidents  (Wiggins  &  O'Hare,  1993),  and  this 
trend  is  not  expected  to  change  in  the  immediate  future  (Diehl,  1991).  Many  of  tiKe 
accidents  have  involved  pilots  pressing  on  into  deteriorating  weather  conditions.  The 
main  causal  factors  in  such  accidents  are  decision  making,  judgement  error  and  pilot 
attitude  (Rockwell,  Roach  &  Griffin,  1981;  Wiggins  &  O'Hare,  1993).  When  these  three 
factors  are  combined  it  can  result  in  an  unrealistic  situation  assessment  (O'Hare,  1990), 
and  a  resulting  accident  or  incident.  However,  decision  skills  may  be  improved 
through  appropriate  training,  thereby  reducing  the  number  of  decision-related 
accidents  (Kaempf  &  Klein,  1993). 

Although  decision  making  has  been  studied  extensively  in  general  psychology  (Beach 
&  Mitchell,  1978;  Einhom  &  Hogarth,  1981;  Gick  &  Holyoak,  1980;  Tversky,  1972), 
much  of  this  research  has  dealt  only  with  single  stimulus  mathematical  and  statistical 
models,  with  little  research  dealing  specifically  with  real  world  environments,  such  as 
aviation.  Until  recently,  most  research  was  based  upon  the  "classic"  analytical  and 
systematic  methods  of  decision  making  such  as  multi-attribute  utility  analysis  (MAUA) 
and  eUmination-by-aspects  (Edwards,  1987;  Tversky,  1972). 

Beach  and  Lipshitz  (1993)  stated  that  real  life  decision  tasks  differ  markedly  from  the 
tasks  for  which  classical,  analytical  decision  theory  was  designed.  Such  analytical 
metihods  provide  training  for  a  single  type  of  decision  only,  where  the  pilot  has  ample 
time  to  generate  multiple  options  and  evaluate  each  of  these  on  known  dimensions  in  a 
static  environment  (O'Hare,  1992;  Kaempf  &  Klein,  1993;  Rouse  &  Valusek,  1993). 
These  analytical  approaches  are  cumbersome  and  time  consuming,  and  are  therefore 
unsuitable  for  most  aviation  decision  and  judgment  tasks. 

Since  the  late  1980' s,  intuitive  decision  making  models  that  are  more  appropriate  for 
aviation  have  been  developed.  Intuitive  processes  are  those  of  low  cognitive  control, 
carried  out  in  the  mind  immediately,  without  deliberation,  reasoning,  or  conscious 
awareness  (Hammond,  1988;  Lipshitz,  1993).  Because  these  intuitive,  automatic  and 
semi-automatic  cognitive  processes  generally  involve  rapid  rates  of  data  processing, 
they  are  often  well  suited  to  time-pressured  environments.  Naturalistic  Decision 
Making  (NDM)  is  a  well-documented,  intuitive  decision  strategy  (Klein,  1989).  Models 
of  NMD  include  Klein's  (1989)  Recognition  Primed  Decision  Malting  (RPD),  in  addition 
to  O'Hare's  (1992)  "ARTFUL"  decision  maker.  These  models  represent  a  new  direction 
in  aeronautical  decision  making,  as  they  take  decision  analysis  out  of  the  sterile, 
predictable  laboratory,  and  examine  it  in  dynamic,  naturalistic  environments  and 
operational  settings,  where  time  is  limited,  goals  are  d5mamic,  data  are  questionable, 
and  decision  consequences  are  high  (Kaempf  &  Klein,  1993). 

The  decision  strategy  chosen  depends  on  the  problem  type  (familiarity,  ambiguity, 
complexity,  stability),  the  environment  (time,  risk,  resources,  problem  significance, 
irreversibility),  and  the  characteristics  of  the  decision  makers  (expert,  novice, 
motivation,  knowledge,  ability)  (Beach  &  Mitchell,  1978;  Hammond,  1988;  Orasanu, 
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1993).  Most  cockpit  situations  involve  a  combination  of  time  constraint,  risk,  dynamic 
goals  and  environment,  and  professional  pilots  (intermediate  and  experienced), 
thereby  making  NDM  and  RPD  strategies  better  suited  than  analytical  methods. 


1.1  Analytical  Decision  Processes 

Anal3^cal  approaches  rely  on  the  careful,  deliberate  analysis  of  cues,  with  the  pilot 
generating  a  wide  range  of  options,  and  concurrently  evaluating  and  comparing 
evidence,  then  choosing  the  optimal  course  of  action.  Because  the  decision  maker 
generates  a  wide  range  of  responses  and  cormter-responses,  these  processes  usually 
require  large  amounts  of  time,  resources  and  cognitive  expenditure.  The  models  and 
tasks  formulated  are  usually  singular,  well  defined  and  straightforward,  as  are  the 
alternatives  (Hammond,  1988;  Rouse  &  Valusek,  1993).  Although  this  type  of  decision 
strategy  has  a  place  for  some  decisions  and  situations,  (e.g.,  buying  a  car,  deciding  on 
employment  candidates),  the  analytical  approach  is  largely  inappropriate  in 
naturalistic  environments,  and  an  intuitive  approach  appears  to  be  more  appropriate. 

Anal5dical  strategies  deteriorate  when  confronted  with  time  pressure.  Klein  and 
Klinger  (1991)  stated  that  even  under  low  time  pressure,  analytical  strategies  require 
extensive  work  and  lack  flexibility.  Several  studies  on  experienced  engineers  found 
that  even  imder  no  time  pressure,  they  still  relied  heavily  on  NDM  type  strategies, 
regardless  of  information  characteristics,  problem  difficulty,  familiarity  and  structure 
(Hammond,  Hamm,  Grasia  &  Pearson  1987;  Klein  1989).  A  study  by  Zakay  and 
Wooler  (1984)  foimd  that  training  in  anal5^cal  methods  of  decision  making  did  not 
improve  the  quality  of  decisions  when  subjects  were  under  time  pressure.  In  fact, 
Hammond  et  al.  (1987)  found  that  intuitive  cognition  can  outperform  analytical 
processes,  and  is  less  likely  to  result  in  extreme  errors.  Beach  and  lipshitz  (1993)  found 
no  evidence  that  classic  or  formal  decision  models  improve  decision  performance. 
Wickens,  Stokes,  Barnett  and  Hyman  (1993)  stated  that  intuitive  strategies  are  less 
prone  to  degradation  under  stress,  because  retrieval  of  familiar  information  from  long 
term  memory  may  be  relatively  immune  to  the  cogmtively  degrading  effects  of  stress. 
Hammond  et  al.  (1987)  also  stated  that  it  is  difficult  for  ambiguous  cues,  inaccuracies, 
dynamic  environments,  conflicting  goals,  group  processes  and  expertise  to  be  factored 
into  analytical  judgment  and  decision  making  process.  Intuitive  decision  theories  (such 
as  NDM)  are  able  to  account  for  all  of  these  variables. 


1.2  Intuitive  Decision  Strategies 

Altitough  intuitive  decision  strategies  have  found  acceptance  in  the  1990's,  little 
previous  research  had  been  conducted  with  such  strategies.  As  late  as  1978,  Beach  and 
Mitchell  (1978)  discussed  non-anal3rtical  (i.e.  intuitive)  processes  as  being  fairly  simple, 
pre-formtdated  rules  such  as,  "eeny  meeny  miney  mo",  flipping  a  coin,  or  well  known 
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sayings  like  "red  sky  at  night,  sailors  delight".  The  use  of  the  term  "non-analytical" 
rather  than  "intuitive"  was  a  common  occurrence  in  many  contemporary  papers. 
Rather  than  giving  an  acceptable  scientific  definition  of  intuitive  strategies,  they  were 
simply  defined  as  being  the  opposite  of  analytical  strategies.  Such  points  highlight  the 
lack  of  serious  research  previously  conducted  on  intuitive  strategies. 

A  similar  situation  is  fotmd  witix  judgement  and  decision  making  in  dynamic  task 
environments.  Hammond  (1988)  stated  that  of  the  many  thousands  of  articles  and 
journals  on  decision  making,  dynamic  situations  (i.e.  naturalistic  enviroiunents)  have 
hardly  been  touched;  virtu^y  all  research  has  focussed  on  static  tasks.  These  points 
help  to  further  emphasise  the  general  progress  that  has  been  made  in  the  field  of 
intuitive  theories  with  NDM  and  RPD  models  since  the  early  1990's. 

NDM  and  RPD  are  descriptive  (rather  than  normative  or  prescriptive)  approaches  to 
decision  making,  as  they  describe  how  people  actually  choose  between  options,  rather 
than  how  they  should  choose  0onsson,  1991;  O'Hare,  1992).  Brehmer  (1990)  stated  that 
naturalistic  decisions  and  environments  must  be  studied  by  descriptive  models,  yet 
according  to  O'Hare  (1992),  there  has  been  little  effort  put  into  developing  descriptive 
approaches  to  aeronautical  decision  making. 

Many  analytical  theories  of  decision  making,  including  most  aeronautical  decision 
making  strategies  have  so  far  used  a  structured,  prescriptive  approach  (e.g.,  DECIDE, 
SADIE,  IMSAFE),  and  thus,  are  concerned  simply  with  inputs  and  outputs  (Maule  & 
Svenson,  1993).  The  correct  choice  or  outcome  indicates  good  judgement,  and  the 
wrong  choice  or  outcome  indicates  poor  judgement.  The  cognitive  process  a  person 
uses  to  reach  these  outcomes  is  ignored,  and  option  generation  is  treated  as  a 
"cognitive  black  box"  (Kaempf  &  Klein,  1993).  Process  approaches  however,  are 
concerned  with  how  decisions  are  made  in  terms  of  the  underljdng  psychological  and 
cognitive  processes  that  produce  the  outcome  (Maule  &  Svenson,  1993;  Orasanu  & 
CormoUy,  1993).  Hence,  process  approaches  can  be  seen  as  exploring  the  cognitive 
black  box.  Process  analysis  of  decision  making  is  much  more  suited  to  the  study  of 
NDM  and  RPD. 


1.3  Cognitive  Continuum  Theory 

The  cognitive  continuum  theory  states  that  intuitive  decision  making  strategies  occupy 
one  end  of  the  cognitive  continuum,  with  analytic  strategies  at  the  other  end,  and 
quasi-rational  decisions  and  judgements  in  the  middle  (Hammond  et  al.,  1987; 
Hammond,  1988, 1993).  As  suggested  by  Figure  1,  NDM  and  RPD  are  at  opposite  ends 
of  the  continuum  to  traditional,  analytical  and  systematic  decision  making  strategies 
(Hammond,  1993).  Once  a  cognitive  process  has  been  located  on  the  cognitive 
continuum,  it  will  be  found  to  interact  in  predictable  ways  with  various  task 
conditions,  such  as  time  period,  cue  characteristics,  cognitive  control,  errors  and  rate  of 
data  processing  (Hammond,  1993;  Hammond  et  al.,  1987).  Knowledge  of  the  locus  of  a 
task  on  the  task  continuum  index  makes  it  possible  to  predict  the  corresponding 
cognitive  processes  of  the  decision  maker  (Hammond,  1988).  For  example,  tasks 
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requiring  large  amounts  of  information  to  be  processed  in  short  time  periods  induce 
intuitive  methods  (Lipshitz,  1993).  Cognitive  activity  can  oscillate  about  the  cognitive 
continuum  between  intuitive  and  analytical  as  the  task  conditions  change. 


Intuitive 

l' 

Quasi  rationality  Non  compensatory 

Analytical 

*1 

NDM/RPD 

elimination-by-aspects 

EBA 

MAUA 

Figure  1.  Cognitive  Continuum  indicating  locations  of  several  decision  making  strategies  and 
examples  of  each. 


The  cognitive  continutun  theory  also  offers  important  predictions  concerning  human 
error.  In  comparison  to  errors  produced  by  perception  and  intuition,  analytical 
processes  produce  fewer  errors,  but  when  diey  do  occur  the  errors  are  more  serious 
(Hammond  et  al.,  1987).  Both  Lipshitz  (1993)  and  Hammond  (1988)  stated  that 
judgements  and  decisions  are  most  accurate  when  the  location  of  cognitive  processes 
on  die  cognitive  continutun  corresponds  to  the  location  of  the  decision  task  on  the  task 
continuum.  Hammond  (1988)  even  suggested  that  operators  who  employ  intuitive 
judgement  in  response  to  task  conditions  designed  to  invoke  intuitive  cognition  will  be 
more  accurate  m  their  judgements  and  decisions  than  those  whose  methods  are 
analytical  in  these  circumstances.  He  stated  that  this  conclusion  runs  cotmter  to 
previous  research  that  had  argued  for  analytical  solutions  in  all  possible  circumstances. 


2.  NDM  and  RPD 

Briefly,  NDM  is  how  real  people  (including  intermediates,  professionals  and  experts) 
make  decisions  in  complex,  real  world  environments  (see  Fig.  2).  Brehmer  (1990)  and 
Klein  et  al.  (1993)  summarised  the  key  features  of  NDM  (and  these  are  easily  contrasted 
to  analytical  decision  making); 

•  NDM  requires  a  continual  series  of  decisions,  rather  than  a  single,  one-off  decision. 
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•  The  decisions  are  interdependent,  with  one  decision  affecting  those  that  occur  after. 

•  The  environment  and  conditions  are  d)mamic,  continually  changing  both 
autonomously,  and  as  a  consequence  of  the  decision  maker's  actions. 

•  The  decision  maker  must  make  choices  and  judgements  in  real  time. 

•  The  goals  and  tasks  are  often  iU  defined,  d5mamic  and  competing. 

•  The  decision  makers  are  usually  knowledgeable,  experienced,  and  professional. 


Figure  2.  Synthesised  jrrocess  model  ofNDM  (Klein,  1993b,  p.  391). 

RPD  is  one  model  of  NDM  involving  decisions  for  which  alternative  courses  of  action 
are  directly  derived  from  the  recognition  of  critical  information  and  prior  knowledge 
(Klein,  Calderwood  &  MacGregor,  1989).  These  alternatives  are  serially  evaluated,  so 
there  is  no  need  for  the  decision  maker  to  compare  options.  In  most  operational 
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environmBnts  the  aircrew  are  experienced  (or  intermediate)  professionals.  Due  to  the 
experience  of  the  decision  maker,  in  many  cases  the  first  or  second  option  generated  is 
a  workable  one,  and  hence  is  chosen.  Decision  makers  compare  current  events  with 
previously  experienced  events  and  known  rules  stored  in  memory,  recognising 
similarities  to  help  them  select  appropriate  reactions,  judgments  and  decisions.  Hence, 
the  name,  recognition-primed  decision  making.  There  are  three  basic  phases  to  RPD 
(Lipshitz,  1993);  1)  situation  recognition,  2)  serial  option  evaluation,  and  3)  mental 
simulation.  These  processes  are  described  in  a  later  section 

An  example  of  RPD  might  be  a  pilot  immediately  recognising  a  pattern  of  instrument 
readings  and  other  conditions  (both  internal  and  external  to  the  aircraft)  as  attributable 
to  carburettor  icing,  without  the  need  to  go  through  a  time  consuming,  analytical 
reasoning  process.  The  cognitive  activity  that  allows  a  person  to  match  events  and 
knowledge  perceived,  with  an  already  established  template,  is  known  as  pattern  or 
feature  matching  (Hammond,  1988).  Such  templates  may  be  instilled  via  training  or 
experience. 


A.  SMPLE  MATCH 

^jEnperteoee  the  Situation^ 


Recogrtllton  has  four  aspficls  I 
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Figure  3.  Klein's  three  RPD  models  of  various  NDM  situations  (Klein,  1993a,  p.  141). 

Klein's  (1993)  RPD  model  of  NDM  (Fig.  3)  includes  several  variants  according  to 
decision  complexity.  The  simplest  case  is  where  a  situation  is  recognised  and  the 
obvious  reaction  is  implemented.  A  more  complex  model  involves  the  decision  maker 
having  to  mentally  simulate  the  reaction  to  evaluate  it.  The  most  complex  case  is  where 
the  mental  simulation  reveals  flaws,  and  a  modification  must  occur.  These  three  cases 
suggest  that  NDM  can  be  used  in  most  decision  events  a  pilot  faces,  as  long  as  the  pilot 
can  match  what  they  are  currently  experiencing  to  what  they  have  previously 
experienced  or  learnt. 
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O'Hare  (1992)  referred  to  NDM  and  RPD  strategies  activating  "if-then"  or  "condition- 
action"  rules.  Once  a  situation  or  condition  has  been  assessed  and  recognised  as  "that 
type",  then  the  appropriate  reaction  is  implemented.  For  example,  IF  condition  x  is 
met,  THEN  implement  action  y.  The  decision  is  recognition  primed,  as  the  pilot  may 
have  previously  encountered  the  situation,  or  something  similar  to  it.  The  previous 
encoimter  may  have  been  in  either  an  operational  or  training  (including  simulator) 
enviromnent.  These  types  of  condition-action  rules  and  relationships  are  relevant  to 
aviation  where  there  are  many  standard  operating  procedures  and  well-practised 
responses  to  situations  and  conditions. 

The  characteristics  of  naturalistic  environments  have  been  described  by  several 
researchers  (Brehmer,  1990;  Kaempf  &  Klein,  1993;  Orasanu  &  Connolly,  1993).  These 
characteristics  are  summarised  in  Table  1,  and  the  key  features  of  NDM  and  RPD 
decision  making  are  summarised  in  Table  2.  Many  of  the  points  listed  are  easily 
contrasted  to  the  components  of  analytical  decision  making. 

Table  1.  A  Summary  of  the  Characteristics  of  Naturalistic  Environments 


•  uncertain,  ambiguous  or  missing  data  -  data  may  even  be  incorrect  or  conflicting. 

•  Ill-defined  and  ill-structured  goals  and  tasks 

•  shifting  and  competing  goals  -  situation  is  rarely  dominated  by  single,  well 
imderstood  goals,  instead  goals  may  be  ill-defined,  incompatible  and  dynamic 

•  uncertain,  dynamic  environments  -  situations  are  not  static,  straightforward  or 
unambiguous 

•  action/feedback  loops  -  a  series  of  events  and  decisions  will  alter  subsequent  goals  and 
actions 

•  time  pressure  -  leads  to  decision  making  involving  heuristics,  biases  and  cognitive 
shortcuts.  Aircraft  may  travel  over  15km  in  1  min,  thus  time  is  crucial 

•  high  stakes  and  risks  -  consequences  of  error  are  high,  even  to  the  point  of  life/ death 

•  multiple  players  and  teams  -  many  aircraft  have  2  or  3  pilots,  as  well  as  engineers  and 
flight  attendants,  in  addition  to  ATC,  other  aircraft,  and  ground  staff  to  be 
considered 

•  organisational  goals  and  norms  -  the  overall  mission  statement  or  goals  of 
organisations  exert  pressure  that  may  conflict  and  bias  the  situation 

•  experienced  decision  makers  -  novices  are  rarely  studied  in  NDM,  as  most 
professionals  are  experienced  or  intermediates. 
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Table  2.  A  Summary  of  the  Key  Features  cfNDM  and  RPD 


•  Serial  generation  and  evaluation  of  options,  rather  than  concurrent  evaluation  and 
deliberation 

•  First  option  chosen  is  usually  reasonable  and  workable,  rather  than  semi-random 
generation  and  selective  retention  of  many  options. 

•  Satisfactory  options  and  outcomes  rather  than  optimal  ones 

•  Evaluation  through  mental  simulation,  rather  than  maths,  statistics  or  comparison 
analysis 

•  Focus  on  situation  assessment,  rattier  than  decision  events  (such  as  option  ranking) 

•  Decision  maker  is  primed  and  committed  to  act,  rather  tiian  waiting  for  a  complete 

analysis  _ 


(Adapted  from  Klein  &  Klinger,  1991;  Klein,  1993) 


2.1  Situation  Assessment  and  Mental  Simulation 

2.1.1  Situation  Assessment 

Decision  tasks  can  be  divided  into  situation  assessment  and  actual  decision  making 
(Kirschenbaum,  1992).  This  is  another  area  where  the  underlying  principles  between 
intuitive  and  analytical  decision  processes  differ.  Orasanu  and  Connolly  (1993) 
reported  that  the  major  factor  that  distinguishes  expert  from  novice  decision  makers  is 
the  experts'  situational  awareness  and  agreement  ability,  not  their  actual  reasoning  and 
decision  processes.  Most  analytical  decision  theories  focus  upon  generating  the  optimal 
option  and  the  actual  selection  of  the  best  alternative,  whereas  NDM  and  RPD 
strategies  focus  upon  the  earlier  stage  of  situation  assessment  with  very  little  time 
spent  generating  alternatives  (Mosier,  1991;  Orasanu  &  Connolly,  1993).  This  is  because 
in  time-pressure  situations,  a  decision  may  need  to  be  made  before  the  nature  of  the 
problem  is  properly  defined  or  imderstood,  hence  the  optimal  decision  outcome  may 
never  be  known.  Furthermore,  dynamic  goals  and  changing  conditions  mean  that  ttie 
optimal  outcome  is  often  dynamic  in  itself. 

Because  naturalistic  environments  are  dynamic,  the  decision  maker  must  continuously 
assess  the  situation  in  order  to  reassess  the  appropriateness  of  situation  models  and 
hypotheses  (Kirschenbaum,  1992).  Thus,  situation  assessment  and  awareness  is  a 
crucial  process  for  intuitive  decision  strategies.  Situation  awareness  is  a  prerequisite  for 
good  decision  making,  correlating  positively  with  decision  accuracy  (Klein,  1989; 
Mosier,  1991;  Noble,  1993).  Once  the  pilot  rmderstands  a  situation  an  acceptable  course 
of  action  is  often  easily  identified  (Kaempf  &:  Klein,  1993). 
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Figure  4  displays  a  cognitive  model  of  the  situation  assessment  stage  of  RPD.  It 
outlines  how  experienced  decision  makers  use  previous  experiences  to  assess  a 
situation  and  identify  appropriate  actions.  Noble  (1993)  stated  that  it  is  dtiring  the 
situation  assessment  phase  that  a  decision  maker  interprets  the  meaning  of  the 
situation,  inferring  the  reasons  why  the  situation  appears  as  it  does,  assesses  the 
inherent  risks  and  opportunities,  and  identifies  the  actions  to  minimise  these  risks  and 
maximise  the  opportunities.  These  actions  are  identified  by  comparing  the  situation 
with  previously  experienced  reference  situations  encoded  m  memory.  Each  reference 
problem  has  general  solution  methods  and  situation  conditions  that  apply  to  it.  The 
reference  problems  that  match  the  current  situation  are  activated  when  tike  problem 
solution  methods  associated  with  the  activated  reference  problem  become  candidate 
actions.  Table  3  describes  the  important  role  that  situation  assessment  has  in  NDM. 


Figure  4.  Noble's  model  of  situation  assessment  in  NDM/RPD.  (Noble,  1989,  cited  in 
Lipshitz,  1993,  p.  106) 


Table  3.  The  Role  of  Situation  Assessment  in  NDM 

*  It  helps  to  imderstand  the  t5^es  of  goals  that  can  realistically  be  accomplished 
given  the  situational  conditions. 

•  Situation  assessment  also  allows  the  decision  maker  to  increase  the  salience  of 
important  cues 

•  It  helps  for  expectations  that  can  serve  as  a  check  on  the  accuracy  of  the  situation 
assessment. 

*  It  allows  typical  actions  to  be  identified 

Adapted  from  Klein  (1993) 
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Therefore,  good  situation  assessment  allows  the  information  available  about  a  situation 
to  be  compared  to  stored  encounters,  and  a  suitable  working  model  of  the  true 
situation  can  be  developed  (Kirschenbaum,  1992). 

Brehmer  (1990)  stated  that  "to  control  a  system,  a  ....  decision  maker  must  have  a  model 
of  the  system  it  seeks  to  control"  (p.265).  Therefore,  effective  judgement  and  decision 
making  cannot  take  place  unless  a  suitable  model  or  hypothesis  has  been  generated. 
When  the  correct  model  or  hypothesis  is  generated  -  and  there  may  be  more  than  one 
available  -  the  pilot  can  then  plan  ahead,  allowing  proceeding  behaviours  and  choices 
to  be  guided  by  expectancy  and  hypotiiesised  schemas  (Kirschenbaum,  1992).  If  an 
incorrect  model  is  developed,  the  results  and  actions  that  follow  can  be  disastrous  (e.g., 
British  Midlands  B737  crash  at  Kegworth,  USS  Vincennes  shootdown  of  friendly 
aircraft,  American  Airlines  B757  crash  at  Cali,  Columbia).  The  RPD  model  (Fig.  1) 
suggests  that  decision  makers  may  fail  due  to  faulty  situation  assessment  and  mental 
simulation  stemming  from  incorrect  models. 

Mosier  (1991)  foxmd  that  many  aircrews  did  not  even  wait  imtil  they  had  a  complete 
vmderstanding  of  their  situation  to  make  and  implement  decisions.  Rather,  they  made 
recognition,  reflexive  judgements  based  upon  a  few  critical  items  of  information. 
Continual  situation  assessment  was  then  used  to  verify  the  decision's  acceptability.  If 
the  course  of  action  needed  to  change,  a  second  option  would  be  generated  and 
implemented  in  the  same  manner.  This  process  emphasises  the  importance  of 
situational  assessment  and  action/ feedback  loops. 


2.1.2  Mental  Simulation 

The  other  important  phase  of  RPD  -  mental  simulation  -  is  the  final  process  carried  out 
before  a  decision  action  is  implemented,  or  during  its  implementation.  The  decision 
maker  acting  out  the  decision  in  his/her  mind,  imaging  how  a  sequence  of  events 
might  unfold  within  a  given  context,  thereby,  evaluating  the  course  of  action  via 
mental  simulation  (Klein,  1993).  The  simulation  often  includes  the  successive  steps  to 
be  taken,  the  potential  outcome  of  these  steps,  the  problems  that  are  likely  to  be 
encountered,  and  how  these  problems  can  be  handled  (Lipshitz,  1993).  Klein  (1993) 
stated  that  mental  simulation  is  generally  used  to  assess  whether  the  decision  maker's 
understanding  or  model  of  a  situation  is  logical,  and  to  determine  what  may  go  wrong 
if  a  decision  action  is  implemented.  As  a  result  of  the  simulation,  the  decision  maker 
either  rejects,  modifies  or  implements  the  action  (as  per  Figure  2).  Because  NDM 
emphasises  satisfactory  rather  than  optimal  decision  outcomes,  the  mental  simulation 
is  generally  a  rapid  procedure,  acting  as  a  go/ no-go  check.  If  time  is  not  adequate  for  a 
complete  mental  simulation,  the  decision  maker  will  simply  implement  the  decision 
action  that  experience  has  generated  as  the  most  likely  to  be  successful  and  make 
subsequent  changes  as  necessary  to  maintain  a  satisfactory  outcome  (Klein,  1993). 

Large  transport  aircraft  usually  have  two  or  three  crew  members  in  the  cockpit.  In  this 
situation,  mental  simulation  is  important  in  ensuring  that  the  crew  members  all  share 
the  same  mental  model  (Orasanu  &  Connolly,  1993).  Only  when  the  crew  members  are 
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all  working  together  on  the  same  problem  can  team  decision  making  be  effective  and 
efficient.  Shared  mental  models  allow  crews  to  function  efficiently  because  they  serve 
as  an  organising  framework,  allowing  the  crew  to  anticipate  events  as  well  as  each 
others  actions;  vital  for  effective  NDM  and  RPD  (Orasanu,  1993).  Shared  mental 
models  can  only  be  achieved  through  good  crew  communication  and  interaction. 


3.  Applied  NDM 

There  have  been  several  studies  of  professionals  in  their  operational  settings  using 
NDM  and  RPD  strategies.  These  include  studies  on  fireground  commanders  (Klein, 
Calderwood  &  Clinton-Cirocco,  1986),  wildland  firefighters  (Taynor,  Klein  & 
Thordsen,  1987),  rotary  wing  aircraft  (Thordsen,  Klein  &  Wolf,  1992),  battle  ship 
commanders  (the  AEGIS  cruiser  commander  -  Kaempf,  Wolf,  Thordsen  &  Klein,  1992), 
tank  platoon  leaders  (Brezovic,  Klein  and  Thordsen,  1987),  and  design  engineers  (Klein 
&  Brezovic,  1986).  The  following  discussion  centres  on  the  fireground  commanders 
(FGCs)  study  because  it  is  thoroughly  docximented  (Klein  et  al.,  1986).  The  findings  of 
all  the  studies  are  very  similar,  supporting  what  has  been  described  in  this  report. 


3.1  The  Fire  Ground  Commanders  (FGCs) 

The  FGCs  reported  that  their  decision  making  processes  did  not  involve  "making 
choices",  "considering  alternatives",  or  "assessing  probabilities".  Rather,  they  stated 
that  they  were  acting  and  reacting  on  the  basis  of  prior  experience;  the  basis  for  their 
decisions  was  the  ability  to  recognise  and  appropriately  classify  situations  (ie., 
situation  assessment).  Wi&  the  use  of  mental  models,  plans  were  generated,  monitored 
and  modified  as  the  situation  and  conditions  changed.  Using  their  experience,  the 
FGCs  usually  generated  a  workable  option  as  the  first  to  come  to  mind.  There  was  no 
extensive  option  generation,  nor  was  there  any  concern  to  generate  the  optimal  choice. 
The  concept  of  serial  evaluation  is  important,  as  tire  FGCs  reported  that  after  mental 
simulation,  if  a  new  decision  or  course  of  action  is  needed,  the  next  most  workable  or 
likely  reaction  is  selected.  This  mental  search  continues  serially  until  a  workable 
solution  is  identified.  Rarely  are  two  decisions  evaluated  concurrently,  and  the  FGCs 
stated  that  the  reason  for  this  is  simply  time  constraint.  In  the  time  it  takes  to  generate 
and  analytically  compare  two  options,  the  situation  may  have  changed,  and  fire  fire 
may  possibly  get  out  of  control.  If  conscious  deliberation  was  entered  into  at  any  stage, 
it  was  in  terms  of  classifying  and  articulating  the  nature  of  the  decision  problem  itself. 
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There  is  continual  emphasis  and  reliance  on  situational  assessment  and  mental 
simulation'.  Once  tihe  FGCs  recognised  it  as  "that  type  of  problem"  or  "that  specific 
problem",  they  could  recall  the  typical  response  to  be  implemented,  demonstrating 
O'Hare's  (1992)  IF-THEN  h3q?othesis.  Due  to  this  recognition  based  reaction,  the  FGCs 
rarely  considered  more  than  one  option  when  making  a  decision.  Once  the  situation 
was  recognised,  it  sensitised  the  decision  maker  to  important  (not  always  salient)  cues, 
and  helped  tihem  build  appropriate  working  models  of  the  situation.  The  FGCs  could 
then  mentally  nm  through  their  plans  and  models  to  evaluate  the  feasibility  of  their 
implementation.  If  any  problems  were  foreseen,  the  option  would  be  modified  or 
rejected  as  discussed  previously. 

Because  the  FGCs  were  continually  assessing  the  situation  and  modifying  their  plans 
as  needed,  they  were  always  primed  to  act  and  react.  The  FGCs  could  respond 
immediately  to  changes  in  the  action/ feedback  loop,  and  unlike  novices,  they  knew 
and  could  accept  when  to  alter  plans.  Because  the  FGCs  did  not  need  to  wait  for 
complete,  analytical  analysis  and  outcomes  of  the  situation,  they  could  respond  rapidly 
to  changes.  This  often  meant  the  difference  between  containing  tiie  fire  or  not. 


4.  Decision  Optimality 

When  emphasising  how  xmsuitable  analytical  methods  can  be  for  aviation.  Beach  and 
Mitchell  (1978)  stated  that  such  methods  can  work  well  when  there  is  ample  time, 
information  and  money  to  execute  the  strategy  -  a  situation  that  pilots  rarely  find 
themselves  in.  The  conflicting,  incompatible  goals  of  aviation  almost  never  allow  for  a 
single,  optimal  decision  choice  to  be  made.  However,  the  optimal  decision  is  usually 
not  necessary,  rather,  a  good  decision  is  just  as  acceptable  and  leads  to  a  similar 
outcome  without  large  amounts  of  time  and  resources  being  (Einhom  &  Hogarth, 
1981). 

Pilots  generally  implement  the  first  workable  decision,  rather  than  comparing  many 
decisions  for  optimality  (Kaempf  &  Klein,  1993;  Klein  &  Klinger,  1991).  There  is  little 
evidence  that  changing  a  decision  from  good  to  optimal  will  result  in  any  more  safety 
or  efficiency  (Orasanu,  1993).  According  to  Beach  and  Mitchell  (1978),  people  choose 
the  decision  making  strategy  that  requires  the  least  resource  investment  for  a 
satisfactory  solution  (not  an  optimal  one).  In  operational  environments  it  is  important 
to  expend  the  least  personal  resources  (also  known  as  cognitive  economy)  as  there  are 
often  more  situational  and  circumstantial  demands  than  the  single  decision  task. 

In  naturalistic  situations  there  is  often  no  optimal  situation  that  can  be  reached.  An 
optimal  decision  outcome  implies  a  single,  well-defined  goal  has  been  stated,  and  the 


'  For  an  excellent  example  of  the  importance  of  situation  assessment  and  mental  simulation,  see 
Kaempf,  Wolfe,  Thorsden  &  Klein's  (1992)  study  of  the  commander  of  an  AEGIS  cruiser 
deciding  whether  or  not  to  engage  enemy  aircraft. 
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situation  remains  stable  and  unchanging.  One  of  the  features  of  dynamic  enviromnents 
is  that  goals  are  dynamic  and  often  competing.  In  aviation,  there  are  many  such 
dynamic,  competing  goals;  company  policy,  customer  comfort  and  convenience,  safety, 
profit  and  timeliness.  In  such  a  situation  it  is  impossible  to  completely  satisfy  all  of 
these  areas,  and  therefore,  an  analytical  strategy  is  wasted. 

Even  the  concept  of  optimal  and  acceptable  outcomes  is  not  totally  applicable  for 
naturalistic  settings,  as  these  concepts  asstune  that  a  single  decision  will  determine  a 
situation's  outcome.  This  may  be  true  in  the  laboratory,  but  in  everyday  situations, 
decisions  are  embedded  in  larger  tasks  or  goals  ttiat  the  decision  maker  is  trying  to 
accomplish  (Orasanu  &  Connolly,  1993).  A  situation  is  usually  made  up  of  many  small 
tasks  and  decisions,  therefore,  making  a  single  decision  is  not  an  end  in  itself,  but 
rather  one  step  on  the  way  to  achieving  an  overall  goal  (Means,  Salas,  Crandall,  Jacobs, 
1993).  Decision  making  becomes  a  matter  of  providing  direction  for  the  continuous 
flow  of  behaviour  and  monitoring  one's  progress  toward  some  goal,  rather  than 
discrete  episodes  involving  choice  dilemmas  (Brehmer,  1990). 


5.  Heuristics  and  Biases 

Intuitive  decision  making  styles  are  used  when  time  is  limited.  Klein  et  al.  (1989) 
suggested  that  NDM  and  RPD  are  valuable  in  operational  settings  where  there  is  little 
time  for  conscious  deliberation,  as  a  considerable  amoimt  of  deliberation  in  NDM 
process  is  semi-  and  sub-conscious.  Such  time  limitations,  coupled  with  semi-  and  sub¬ 
conscious  processes  result  in  heuristics  and  biases  entering  the  decision  making 
process.  Heuristics  and  biases  cause  people  to  sample  and  process  information  in  a 
selective,  subjective  manner  (Evans,  1990).  Hetuistics  are  quick  and  intuitively  sensible 
rules  of  thumb  tiiat  allow  people  to  summarise  information  and  situations  rapidly  with 
their  oirrent  knowledge  (Bootzin,  Bower,  Crocker  &  HaU,  1991).  Biases  are  often  a 
result  of  heuristics,  and  are  predispositions  on  the  part  of  the  individual  to  respond  in 
certain  ways  (Jonsson,  1991). 

The  importance  of  heuristics  in  making  inferences  has  long  been  recognised.  People 
rely  on  heuristics  to  reduce  the  complex  task  of  assessing  probabilities  and  predicting 
values  (Einhom  &  Hogarth,  1981;  Tversky  &  Kahneman  1974).  In  demonstrating  the 
importance  of  heuristics  to  aviation  decision  making,  Orasanu  (1993)  stated  "cockpit 
decisions  are  heuristics"  (p.  139).  In  naturalistic  environments,  heuristics  may  be  both 
less  effortful  and  more  accurate  then  classical  decision  models  (Cohen,  1993a).  Tversky 
and  Kahneman  (1974)  stated  that  heuristics  are  quite  useful  and  highly  economical,  but 
can  sometimes  lead  to  systematic  and  predictable  errors.  Such  errors  may  include 
sampling  only  salient  data,  ignoring  data  that  conflicts  with  initial  situation 
assessment,  and  basing  situation  assessment  and  diagnosis  on  recently  occurring 
events  because  they  "come  to  mind"  easily. 
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Despite  these  known  errors,  the  complex  nature  of  naturalistic  environments  results  in 
a  person  having  two  alternatives  -  either  build  optimal  models  by  making 
environmental  simplifications,  or  build  heuristic  models  that  maintain  environmental 
realism,  yielding  adequate  rather  than  optimal  decisions  (Einhom  &  Hogarth,  1981). 
Often  the  better  of  the  two  choices  is  to  use  the  cognitively-efficient  heuristics. 
According  to  Einhom  and  Hogarth  (1981),  heuristics  exist  because  they  serve  useful 
functions,  but  above  all  else,  their  benefits  outweigh  their  cost.  However,  heuristics  and 
biases  are  not  cognitive  processes  that  a  person  can  choose  to  implement  or  not.  They 
are  engaged  on  a  sub-conscious,  automatic  level.  Thus,  by  its  very  nature,  the  human 
reasoning  process  is  error-prone  and  sub-optimal  (Cohen,  1993,). 

Some  of  the  more  well-documented  heuristics  and  biases  include  (Evans,  1990;  Mosier- 
O'Neill,  1989;  Tversky  &  Kahneman,  1974;  Wickens,  1985); 

•  the  representative  heuristic  *  the  confirmation  or  anchoring  bias 

•  cue  reliability  or  the  "as  if"  heuristic  *  negative  evidence 

•  cue  salience  or  vividness  *  the  availability  heuristic 

A  well-documented  cognitive  shortcut  that  pilots  use  in  NDM  is  data  sunplification, 
where  technical  data  is  summarised  into  a  few  simple  statements.  This  data 
simplification  is  another  form  of  cogrutive  efficiency.  Studies  by  Layton  and  McCoy 
(1989),  Cohen  (1987),  Kuipers,  Moskowitz  and  Kassirer  (1988),  and  Beck  (1987,  cited  in 
Wiggins  &  O'Hare,  1993)  have  all  revealed  similar  findings  regarding  decision  makers 
in  naturalistic  environments.  The  decision  task  is  often  simplified  by  focussing  on  a 
few  select  aspects  of  tiie  information  available  to  the  pilot  (O'Hare,  1992).  This  is 
neither  a  good  nor  bad  situation,  rather,  it  is  an  effective  simplification.  Whilst  the 
optimal  outcome  will  probably  not  be  realised,  an  acceptable  outcome  can  be  reached 
with  much  less  resoiuce  expenditure  and  time  wastage.  Time  is  a  highly  valued 
resource  in  most  in-flight  decision  scenarios. 

Layton  and  McCoy  (1993)  gave  tiie  following  example  of  how  pilots  focus  on  select 
aspects  of  data  and  information.  Rather  than  a  detailed  imderstanding  of  weather 
situations,  pilots  tend  to  sum  up  the  situation  in  terms  of  positives  or  negatives  at  a  few 
salient  points  en-route.  The  pilots  are  then  attracted  by  the  positives  and  repelled  by 
the  negatives.  According  to  O'Hare  (1992),  tiie  risk  of  pressing  on  into  deteriorating 
weather  may  be  perceived  in  terms  of  "fairly  low",  or  "much  better  than"  the  risks  of  a 
precautionary  search  and  landing.  Such  terms  better  capture  the  nature  of  real  life 
uncertainty  and  ambiguity  that  is  characteristic  of  NDM.  Pilots  prefer  not  to  think 
about  uncertainty  or  ambiguity  in  numerical  or  probabiLLstic  terms,  but  prefer  to  deal 
with  a  single  concrete  representation  of  the  situation  (Cohen,  1987).  Pilots  often  prefer 
qualitative  to  quantitative  reasoning  to  aid  in  NDM  and  RPD  situations  (O'Hare,  1992). 
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6.  Expert  &  Novice  Use  of  NDM 

Good  decision  making  is  an  acquired  skill  (Kirschenbaum,  1992).  NDM  and  RPD  are 
closely  linked  with  the  expert-novice  relationship  because  the  basis  to  such  decision 
strategies  is  intuitive  cognition  and  memory.  A  person  uses  prior  experience, 
encoimters  and  knowledge  to  help  them  deal  with  the  current  judgement  decision.  The 
decision  making  process  is  relatively  rapid,  as  either  the  current  situation  is  in  some 
way  recognisable  to  another,  or  it  is  not.  According  to  Hintzman  (1986),  each  event 
experienced  produces  a  separate  memory  trace,  with  knowledge  derived  from  the  pool 
of  episodic  memory  traces  at  the  time  of  retrieval.  A  retrieval  cue  contacts  all  the  traces 
simultaneously,  activating  those  that  are  similar  to  the  cue.  The  information  retrieved 
from  memory  reflect  the  summed  content  of  all  activated  traces.  A  novice  or 
inexperienced  pilot,  no  matter  how  relatively  skilled  they  are,  simply  will  not  have 
enough  templates  of  situations,  encoimters  or  circumstances  in  memory  to  pattern 
match  current  situations.  Thus  a  novice's  use  of  NDM  or  RPD  is  limited. 

The  importance  of  experience  and  expertise  in  NDM  and  problem  solving  has  been 
well  documented  (Kirschenbaum,  1992;  Klein  et  al.,  1989;  Lipshitz,  1993;  Means  et  al., 
1993;  Mosier,  1991;  Orasanu  &  Connolly,  1993;  Wickens,  1987;  Wickens,  Stokes,  Barnett, 
&  H)Tnan  1993).  Summed  briefly,  these  studies  have  revealed  differences  between 
experts  and  novices  in  how  problems  are  interpreted,  what  strategies  are  devised,  what 
information  is  used,  situation  understanding,  memory  recall  for  critical  information, 
use  of  heuristics,  plausible  goal  setting,  recognising  inappropriate  reactions,  and  speed 
and  accuracy  in  decision  making  and  problem  solving.  Generally,  experts  can  identify 
underl3dng  causes  to  problems  and  have  deeper,  more  complex  models  and 
imderstanding  of  the  problem  and  environment.  However,  the  biggest  difference 
between  experts  and  novices,  especially  in  terms  of  NDM,  is  their  ability  to  assess  and 
evaluate  the  situation,  rather  than  their  ability  to  generate  and  choose  among  options 
(Orasanu,  1993). 

6.1  Information  and  Cue  Sampling 

When  discussing  experience  and  NDM,  information  and  cue  sampling  are  one  area 
that  experts  and  novices  differ  in.  The  human  memory  and  attention  system  are 
limited,  and  deteriorate  when  overloaded  (Wickens,  1987).  Therefore,  when  a  person 
samples  too  much  information,  it  degrades  ratiher  than  improves  decision  making  (eg. 
Omodei  &  Wearing,  1997).  Experienced  decision  makers  know  how  much  information 
they  need  to  make  a  correct  situation  assessment  and  following  suitable  decision.  The 
experienced  decision  maker  needs  only  to  gather  enough  information  to  recognise  the 
situation  as  similar  to  a  previous  encounter  or  not  (Wiggins  &  O'Hare,  1993).  Better 
information  gathering  strategies  lead  to  better  situational  imderstanding,  which  in  turn 
lead  to  better  decision  making.  Inexperienced  pilots  may  waste  time  and  effort 
examining  as  much  information  as  possible.  Much  of  this  sample  information  is 
relatively  imimportant,  as  novice  pilots  are  not  bounded  by  the  experience  of  limited 
previous  outcomes.  Inexperienced  pilots  tend  to  focus  ttieir  search  on  the  most  salient 
cues,  not  necessarily  the  most  important  cues.  In  contrast,  experienced  pilots  are  able  to 
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seek  critical  information  (not  necessarily  ttie  most  salient),  generating  a  small  set  of 
plausible  diagnoses,  options  and  h3^otheses  (Lipshitz,  1993;  Orasanu  &  Connolly, 
1993).  Therefore,  experts  tend  to  use  a  smaller  quantity,  but  greater  quality  of 
information  meaningful  to  good  situation  assessment  and  decision  making 
(Kirschenbaum,  1992). 

Experts  use  more  focussed  information  searches  associated  with  better  structured 
knowledge  schemas.  A  study  by  Beck  (1987,  cited  in  Wiggins  &  O'Hare,  1993) 
compared  weather  related  decision  making  in  experienced  and  inexperienced  pilots. 
The  experienced  pilots  possessed  more  effective  search  strategies,  and  were  more 
accurate.  Klein  (1993)  found  that  experienced  people  generally  used  recognition- 
primed  methods  to  retrieve  a  single  likely  option,  whereas  novices  were  more  likely  to 
use  an  analytical  approach,  systematically  comparing  multiple  options.  Much  of  the 
evidence  in  this  report  suggests  that  the  novice's  approach  to  decision  making  is 
cognitively  inefficient  and  does  not  produce  a  more  effective  outcome. 


6.2  Procedural  Knowledge 

Expertise  in  pilot  decision  making  is  heavily  related  to  procedural  knowledge  (Wickens 
et  al.,  1993).  Such  decision  making  is  also  related  to  Rasmussen's  (1986)  skill-based  and 
rule-based  knowledge.  Beach  and  Mitchell  (1978)  stated  that  a  person's  knowledge  (or 
lack  of  knowledge)  has  the  greatest  influence  on  the  decision  strategy  selection  and  the 
success  of  the  chosen  strategy.  They  also  emphasised  the  importance  of  the  ability  to 
exercise  this  knowledge.  These  two  characteristics  of  the  decision  maker  display  the 
importance  of  experience  or  expertise  to  intuitive  decision  making  processes.  Lipshitz 
(1993)  reiterated  this  by  stating  "The  RPD  model  imderscores  ^e  crucial  role  of 
domain  specific  knowledge  or  experience  in  proficient  decision  making;  no  step  in  the 
(RPD)  model  can  be  executed  effectively  without  such  knowledge"  (p.  109). 

Like  many  other  complex,  high  risk,  socio-technical  systems,  aviation  is  boimd  by 
many  procedures,  rules,  regulations,  standard  operating  procedures  (SOPs),  and  pre¬ 
planned  responses.  For  many  decision  and  judgement  tasks,  a  SOP  or  regulation 
prescribes  the  appropriate  responses,  instantly  limiting  five  or  six  choices  to  one  or 
two.  It  is  the  experts'  well-developed  procedural  knowledge  that  allows  them  to  recall 
all  the  regulations  and  SOPS  that  effect  a  decision,  rapidly  limiting  the  choices 
available,  and  hence,  rapidly  reducing  cognitive  expenditure.  Experienced  aircrew  can 
therefore  solve  problems  faster  by  considering  fewer  alternative  solution  paths. 

In  contrast,  a  novice  pilot  has  only  rudimentary  procedural  knowledge,  and  will  spend 
a  greater  amotmt  of  time  recalling  and  appl3dng  all  relevant  regulations  and  SOPS. 
Therefore,  in  novices,  the  initial  cues  may  not  trigger  a  recall  of  a  manageable  number 
of  possible  schemas  or  rules  (Kirschenbaum,  1992),  causing  a  possible  information 
overload  and  sense  of  confusion,  "paralysing"  tire  inexperienced  decision  maker 
(Lipshitz,  1993).  Again,  this  emphasises  ^e  importance  of  situation  assessment  in 
highlighting  the  appropriate  course  of  action.  However,  if  the  incorrect  problem  or 
course  of  action  is  diagnosed,  then  the  incorrect  SOP  or  regulation  will  be 
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implemented,  leading  to  a  potentially  fatal  situation  (Kaempf  &  Klein,  1993).  Such  was 
the  case  with  the  Kegworth  disaster,  where  a  British  Midlands  B737  aircrew  shut  down 
the  wrong  engine  due  to  incorrect  situation  assessment.  SkiU-based  knowledge  was 
used  to  apply  the  weU-known  rules  and  procedures  for  engine  shutdown.  The  correct 
engine  shutdown  SOP  was  implemented,  but  it  was  based  on  an  incorrect  decision  due 
to  an  incorrect  situation  awareness.  The  aircraft  eventually  crashed  a  few  miles  short  of 
the  runway. 

As  has  been  shown  by  several  researchers  (Chase  &  Simon,  1973;  deGroot,  1966; 
Kirschenbaum,  1992;  Newell  &  Simon,  1972),  experts  tend  to  view  displays,  events  or 
situations  in  terms  of  recognisable  sets  or  patterns  and  learn  to  associate  set  moves, 
reactions  and  procedures  with  each  pattern.  These  patterns  easily  come  to  mind, 
negating  the  need  to  go  through  random  search-and-test  processes  for  all  possible 
solutions  (Means  et  al.,  1993).  This  pattern  further  reduces  the  workload  required  to 
gather  and  process  situational  information,  and  allows  for  rapid  reactions.  Again,  this 
highlights  the  importance  of  situational  assessment  and  awareness  to  NDM. 

Expertise  is  highly  domain  specific,  and  does  not  transfer  easily.  A  chess  grand  master 
uses  RPD  to  choose  a  move  from  tens  of  thousands  of  recognisable  sets  or  patterns  of 
chess  pieces,  often  in  less  than  a  second  (deGroot,  1966).  However,  such  judgement  and 
decision  making  skills  only  confer  an  advantage  on  problems  that  are  meaningful 
within  the  expert's  domain.  Often  skills  acquired  in  one  domain  are  highly  intra¬ 
domain  specific,  and  only  become  generalised  following  extensive  experience  (Wiggins 
&  O'Hare,  1993).  Perferto,  Bransford  and  Franks  (1983)  found  that  after  inexperienced 
subjects  had  acquired  knowledge  in  a  specific  situation,  they  failed  to  transfer  the 
information  to  a  novel  situation,  even  though  the  two  situations  had  similar 
characteristics.  Unless  a  pilot  is  able  to  connect  the  intra-domain  knowledge,  it  makes 
NDM  and  RPD  difficult. 


7.  NDM  Instruction 

This  document  has  continually  emphasised  the  important  relationship  between  NDM 
and  operator  experience.  This  relationship  has  implications  if  naturalistic,  intuitive 
decision  strategies  are  to  be  taught  to  inexperienced  or  novice  pilots.  Despite  the 
problems  of  this  imphcation,  Htde  has  been  done  in  formulating  techniques  and 
strategies  to  teach  intuitive  decision  processes.  The  topic  has  been  treated  rather  like 
that  of  weather-related  decision  making  for  pilots,  that  is,  it  is  assumed  that  the  trainee 
will  eventually  learn  through  experience.  Unfortunately,  learning  by  experience  is  not 
always  an  effective,  efficient,  practical  or  safe  practise  in  aviation  (Simpson  &  Wiggins, 
1997).  Several  authors  have  briefly  covered  areas  of  NDM  tiiat  could  and  should  be 
taught  to  inexperienced  pilots,  but  literature  and  ideas  on  how  this  cotdd  be 
accomplished  is  very  hmited.  There  are  two  possible  ways  to  improve  NDM;  decision 
training  and  decision  aiding.  The  following  section  examines  these  two  options, 
although  it  concentrates  on  decision  training  rather  than  aiding. 
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7.1  Decision  Aiding 

Decision  making  aids  in  the  form  of  computers  can  help  decision  makers  by  computing 
and  depicting  situational  features  that  are  important  for  selecting  a  course  of  action 
(Noble,  1993).  Decision  aids  focus  on  improving  operator  situation  awareness  because 
of  the  importance  of  this  concept  to  NDM.  Decision  aids  help  the  pilot  by  increasing 
the  salience  of  the  most  significant  aspects  of  the  situation  for  its  correct  interpretation. 
The  aids  may  also  help  by  suggesting  several  effective  courses  of  action.  A  decision 
aiding  system  shotdd  not  become  a  decision  making  system,  and  it  should  never 
simply  dictate  decision  courses  to  the  operator.  Rather,  decision  aiding  can  help  less 
experienced  pilots  interpret  a  situation  more  as  an  experienced  pilot  would,  by 
selecting  and  presenting  the  important  information  that  an  experienced  operator 
would  normally  interrogate.  Decisions  aiding  can  also  help  an  experienced  pilot  under 
stress  interpret  a  situation  more  as  they  would  when  not  under  stress,  by  presenting 
specific  information,  rather  than  sunply  a  large  amount  of  data.  If  decision  aids  can 
help  improve  a  pilot's  situational  awareness,  then  they  will  definitely  aid  in  NDM. 

Humans  are  inaccurate  at  estimating  probabilities,  and  are  particularly  poor  at  revising 
probabilities  based  on  new  information  (Mosier,  1991).  Decision  aids  can  be  used  to 
help  improve  this  fault  in  decision  making.  O'Hare  (1992)  reports  on  the  "pilot's 
associate",  a  decision  aid  for  military  pilots,  consisting  of  both  situation  assessment  and 
planning  modules.  It  uses  artificial  intelligence  to  model  the  decision  processes  of 
pilots  to  increase  the  overall  system  performance.  However,  whilst  such  decision  aids 
may  be  installed  into  fighter  aircraft  or  battleships,  due  to  size,  weight  and  cost,  they 
may  not  be  feasible  to  install  in  light  aircraft  or  small  commercial  aircraft.  It  is  these 
aircraft  that  are  often  flown  by  the  least  experienced  pilots,  and  hence  are  the  pilots 
that  would  most  benefit  from  the  technology. 


7.2  Decision  Training 

Whilst  military  platforms  and  large  commercial  aircraft  may  afford  the  luxury  of 
decision  aiding  devices,  it  is  impractical  for  small,  general  aviation  aircraft  to  have 
powerful  decision  aiding  computers  on  board.  Because  of  this,  pilot  traming  is  a  good 
alternative.  According  to  both  Means  et  al.  (1993)  and  Orasanu  (1993),  there  is  no 
evidence  to  suggest  that  it  is  possible  to  improve  all-purpose  decision  making  skills. 
Rather,  specific  components  and  skills  need  to  be  developed. 

Several  authors  have  discussed  the  validity  of  teaching  "debiasing"  techmques  to  pilots 
that  impart  awareness  regarding  the  sorts  of  heuristics  and  biases  that  effect  the 
decision  process  (Wickens,  1987).  However,  Means  et  al.  (1993)  argued  that  in 
debiasing  the  pilot,  the  trainer  is  extinguishing  the  natural  and  intuitive  means  of 
thinking,  which  is  the  basis  of  NDM.  Reducing  the  use  of  heuristics  and  biases  has 
proven  to  be  hard  to  accomplish,  and  previous  attempts  have  not  been  very  successful, 
tending  to  be  of  limited  generality  and  duration  (Cook,  Angus  &  Campbell,  1999). 
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Westerlimd  (1991)  reported  that  the  attitudinal  and  motivational  dimensions  of  a  pilot 
affect  aU  decisions,  thus  training  in  this  region  may  assist  the  decision  process.  Such 
motivations  and  attitudes  encompass  the  "five  hazardous  thought  patterns"  of  macho, 
invulnerability,  resignation,  impulsivity  and  anti-authority  (Buch  &  Diehl,  1984).  If 
pilots  learn  to  control  these  hazardous  attitudes  then  decision  making  may  be  more 
efficient  and  safe.  Lester  and  Bombaci  (1984)  stated  that  these  hazardous  attitudes 
cause  pilots  to  make  decisions  that  are  (in  hindsight)  inappropriate,  counter-intuitive 
and  often  defy  common  sense. 

Several  researchers  have  discussed  die  importance  of  metacognition  and  self- 
evaluation  training  to  improve  NDM  (Klein,  1993;  Means  et  al.,  1993;  Orasanu,  1993). 
This  is  closely  related  to  pilot  awareness  of  hazardous  attitudes,  as  metacognition 
training  may  allow  pilots  to  become  more  aware  and  regulative  of  their  cognitive 
processes  and  resulting  behaviours.  Learning  to  monitor  decision  making  processes 
may  mean  that  pilots  are  better  able  to  manage  time  and  realise  when  they  have  a 
workable  solution,  negating  the  need  to  search  further.  Although  only  speculative,  it 
could  be  suggested  that  metacognition  training  may  help  inexperienced  pilots  from 
getting  bogged  down  with  information  and  options,  allowing  for  faster,  more  intuitive 
decision  processes. 

Westerlimd  (1991)  discussed  the  importance  of  information  processing  and  recall  of 
stored  events  in  NDM.  Jonsson  (1991)  discussed  how  relatively  poor  pilots  are  at 
revising  decisions,  and  Senders  and  Moray  (1990)  suggested  that  pilots  need  training  in 
"how  to  change  one's  mind"  to  "avoid  cognitive  lockup".  Metacognitive  training  may 
help  improve  areas  such  as  these,  as  it  would  allow  pilots  to  be  more  aware  of  their 
capabilities  and  limitations.  Pilots  with  good  metacognitive  skills  have  an  overall  better 
idea  of  their  strengths  and  weaknesses,  and  potential  problems,  and  hence  can  factor 
this  into  NDM  to  be  more  effective  and  flexible  (Orasanu,  1993).  The  advantage  of 
teaching  metacognitive  skills  to  pilots  is  that  they  are  "the  most  trainable  decision 
related  skill  complex"  (Orasanu,  1993,  p.  106),  and  are  "the  best  candidate...that  will  aid 
decision  making"  (Means  et  al,  1993,  p.  324).  There  is  a  sizeable  literature  on  training 
metacognitive  skills  (Means  et  al.,  1993).  According  to  Means  et  al.  (1993),  such  skills 
must  be  taught  in  the  context  of  practising  domain-relevant  decisions;  general 
metacognitive  skills  cannot  be  taught  in  isolation,  as  students  do  not  learn  how  to 
incorporate  these  skills  into  NDM  situations.  Once  pilots  do  understand  how  to  be 
metacognitive  in  specific  situations,  they  can  start  to  apply  the  skill  to  general 
situations. 

Training  in  situational  assessment  and  mental  simulation  is  of  prime  importance  to 
NDM.  Several  studies  cited  in  O'Hare  (1992)  described  how  pilots  "lack  the  big 
picture"  (p.  7),  and  suffer  from  "route  tunnel  vision"  (p.  7)  as  well  as  having 
demonstrated  "widespread  deficiencies  in  acquiring  and  interpreting  ...  information" 
(p.  7).  Such  findings  emphasise  the  idea  that  situation  awareness  skills  are  lacking  in 
inexperienced  pilots  and  need  to  be  improved.  Although  most  researchers  agree  with 
this,  few  suggest  how  it  should  be  done.  Orasanu  (1993)  suggested  that  situational 
awareness  could  be  improved  with  considerable  pattern  recognition  practise,  and  the 
development  of  mental  models  of  the  aircraft  and  related  systems.  She  stated  that 
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crews  must  be  trained  to  rapidly  recognise  situational  patterns  that  trigger  stored 
memory  templates  of  events,  and  pilots  must  then  learn  the  response  side  of  the 
pattern.  For  example,  an  engine  failure  in  a  light  twin  is  met  by  an  almost  automatic 
pilot  reaction  (an  example  of  RPD)  due  to  the  continual  practice  of  it.  Crews  need  to  be 
trained  to  pay  attention  to  ambiguous,  tmexpected  or  abnormal  cues,  as  these  often 
lack  saliency,  and  could  hold  vital  information  for  situation  assessment  and  recognition 
(Orasanu,  1993). 

Mental  simulation  requires  teaching  to  enable  pilots  to  anticipate  future  events  and 
decision  outcomes,  as  well  as  critiqueing  their  own  plans.  From  training  in  mental 
simulation,  pilots  may  be  able  to  learn  to  accept  that  their  plans  may  need 
modification,  or  completely  new  plans  may  need  implementing.  Closely  tied  in  with 
the  teaching  of  situation  assessment  and  mental  simulation,  is  the  teaching  of  shared 
mental  models.  By  teaching  aircrews  effective  communication  and  interaction  skills, 
shared  mental  models  may  be  developed  more  effectively,  allowing  for  better  NDM. 

Kaempf  and  Klein  (1993)  described  several  other  skills  that  may  need  training  in  pilots, 
including  risk  assessment  and  resource  management.  Pilots  need  to  imderstand  that 
safety  is  of  paramount  importance,  and  risks  need  to  be  minimised.  This  ties  in  with 
metacognition  and  hazardous  attitude  awareness  and  training.  Orasanu  (1993)  stated 
that  resource  management  needs  to  be  taught  to  pilots,  as  it  impacts  on  the  quality  of 
decisions  they  make.  Resource  management  is  closely  related  to  metacognition,  as 
effective  resource  management  can  help  crews  reduce  the  demands  on  their  own 
cognitive  resources,  especially  at  times  of  high  workload.  Effective  resource 
management  allows  for  flexibility  in  situations,  which  is  an  important  part  of  NDM. 
Teacli^g  resource  management  involves  several  things  including,  imparting  an 
understanding  in  pilots  of  what  must  be  done  in  most  decision  situations,  how  to 
identify  and  use  all  available  resources,  how  to  manage  time  and  resources  (including 
crew)  effectively,  and  an  understanding  of  how  to  prioritise  tasks  (Orasanu,  1993). 
These  features  of  resource  management  are  very  similar  to  what  is  taught  in  crew 
resource  management  courses  (CRM)  and  hence,  the  teaching  of  this  area  of  NDM  can 
be  modelled  from  the  CRM  courses. 

The  general  consensus  of  all  the  above  mentioned  training,  is  that  for  maximised 
effectiveness  it  should  be  taught  to  pilots  in  naturalistic  enviromnents,  ie,  time 
pressure,  high  workload,  uncertainty,  dynamic,  otiier  people  involved  etc. 


8.  Conclusion 

It  has  been  determined  through  several  studies  that  experienced  operators  (including 
pilots)  in  their  operational  settings  make  many  decisions  using  intuitive  rather  than 
analytical  strategies.  NDM  is  one  such  strategy  in  these  naturalistic  environments,  and 
RPD  is  one  of  the  most  widely  described  models  of  NDM.  Naturalistic  strategies 
describe  how  experienced  people  make  decisions  imder  conditions  of  time  pressure. 
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dynamic  environments  and  goals,  tmcertain  cues  and  high  risk.  All  of  these 
characteristics  are  found  in  aviation  environments. 

The  earlier  sections  of  this  report  examined  the  principles  of  NDM,  detailing  theoretical 
perspectives  that  explain  how  NDM  and  RPD  work,  and  how  such  skills  are  acquired, 
including  how  the  situation  recognition  actually  occurs.  Intuitive  and  analytical 
decision  making  strategies  were  compared  and  contrasted,  highlighting  the 
appropriateness  of  intuitive  methods  (NDM  and  RPD)  to  operational  settings  such  as 
aviation. 

The  key  features  of  NDM  and  RPD  were  examined.  A  discussion  of  the  RPD  model 
demonstrated  the  importance  of  situational  assessment  and  mental  simulation  to 
NDM.  The  importance  of  expertise  and  experience  was  a  theme  running  throughout 
the  report,  and  the  differences  between  expert  and  novice  decision  making  were 
presented.  It  would  seem  that  only  experienced  pilots  would  be  capable  of 
implementing  RPD  due  to  the  reliance  on  memory  traces  and  pattern  recognition. 

The  final  section  of  this  document  addressed  decision  aiding  and  instruction,  and  areas 
of  NDM  and  RPD  that  may  be  able  to  be  taught  to  novices  and  inexperienced  pilots 
were  examined.  Although  there  has  been  little  work  done  to  formulate  teaching 
strategies  for  NDM  and  RPD,  some  ideas  were  presented.  It  appears  possible  to  teach 
novice  pilots  at  least  the  basics  of  NDM,  which  they  can  then  develop  themselves. 
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